Search engines discover and store information about documents such as web pages, which they typically retrieve from the code of the documents. The documents are sometimes retrieved by a crawler or an automated browser, which may follow links in a document or on a website. Conventional crawlers typically analyze documents as flat text files examining words and their positions (e.g. titles, headings, or special fields). Data about analyzed documents may be stored in an index database for use in later queries. A query may include a single word or a combination of words.
Many documents include graphical, video, audio, and similar elements in addition to textual data. Links between documents (e.g. hyperlinks to other documents) are also a common occurrence. Furthermore, documents such as word processing documents or spreadsheets may contain externally sourced data that may be shared between multiple documents. Moreover, embedding whole documents in other documents is also regularly used.
Flat text based searches may not necessarily capture multi dimensional relationships among documents and between documents and data sources such as those described above. To capture relationships between linked/embedded documents or documents and data sources, multiple searches may have to be performed increasing use of system resources and/or losing relationship information in the rendering of the search results.